Reinforcement Learning in Complex Environments Through Multiple Adaptive Partitions
نویسندگان
چکیده
The application of Reinforcement Learning (RL) algorithms to learn tasks for robots is often limited by the large dimension of the state space, which may make prohibitive its application on a tabular model. In this paper, we introduce LEAP (Learning Entities Adaptive Partitioning), a model-free learning algorithm that uses overlapping partitions which are dynamically modified to learn near-optimal policies with a small number of parameters. Starting from a coarse aggregation of the state space, LEAP generates refined partitions whenever it detects an incoherence between the current action values and the actual rewards from the environment. Since in highly stochastic problems the adaptive process can lead to over-refinement, we introduce a mechanism that prunes the macrostates without affecting the learned policy. Through refinement and pruning, LEAP builds a multi-resolution state representation specialized only where it is actually needed. In the last section, we present some experimental evaluation on grid worlds, from [10].
منابع مشابه
Learning in Complex Environments through Multiple Adaptive Partitions
When using tabular value functions, the application of Reinforcement Learning (RL) algorithms to real-world problems may have prohibitive memory requirements and learning time. In this paper, we introduce LEAP (Learning Entities Adaptive Partitioning), a novel model-free learning algorithm in which the state space is decomposed into several overlapping partitions which are dynamically modified ...
متن کاملIndirect Reinforcement Learning with Adaptive State Space Partitions
Model-based reinforcement learning can be applied to problems with continuous state spaces by discretizing the spaces with crisp or fuzzy partitions. The manual definition of suitable partitions, however, is often not trivial, since fine partitions lead to a high number of states and thus complex discrete problems, whereas coarse partitions can be unsuitable for the representation of the optima...
متن کاملMotivated Learning from Interesting Events: Adaptive, Multitask Learning Agents for Complex Environments
This paper presents a model of motivation in learning agents to achieve adaptive, multi-task learning in complex, dynamic environments. Previously, computational models of motivation have been considered as speed-up or attention focus mechanisms for planning and reinforcement learning systems, however these different models do not provide a unified approach to the development or evaluation of c...
متن کاملDynamic Obstacle Avoidance by Distributed Algorithm based on Reinforcement Learning (RESEARCH NOTE)
In this paper we focus on the application of reinforcement learning to obstacle avoidance in dynamic Environments in wireless sensor networks. A distributed algorithm based on reinforcement learning is developed for sensor networks to guide mobile robot through the dynamic obstacles. The sensor network models the danger of the area under coverage as obstacles, and has the property of adoption o...
متن کاملAn Adaptive Architecture for Modular Q-Learning
Reinforcement learning is a technique to learn suitable action policies that maximize utility, via the clue of reinforcement signals: reward or punishment. Q-learning, a widely used reinforcement learning method, has been analyzed in much research on autonomous agents. However, as the size of the problem space increases, agents need more computational resources and require more time to learn ap...
متن کامل